The problem of camera shake or camera jitter can be alleviated through the use of dedicated video capture devices at the client's side, like for instance Microsoft Video Kinect, or systems based on Time-Of-Flight (TOF) cameras. Such systems however are not portable and they involve a dedicated set-up procedure that does not enable ubiquitous access, i.e. access at any time from any location, to immersive videoconferencing.
Through the use of mobile devices, ubiquitous access to video conferencing has become possible. Through the increased use of mobile devices in video conferencing however, the camera shake or camera jitter problem is encountered which generally degrades the output quality of the foreground extraction process and consequently also the quality of the immersive video streams that are generated from the extracted foreground images. Reliable detection of camera shake in order to enable taking appropriate corrective and/or preventive measures improving the quality of immersive video hence is desirable.
One category of solutions makes use of device specific hardware such as accelerometers present in certain mobile phones, tablet-PCs, etc. in order to detect camera shake. These solutions however rely on the presence of dedicated hardware and are therefore not device-agnostic. Further, these solutions leave a range of portable devices that is typically not equipped with such hardware, like for instance laptops, uncovered.
Another category of solutions relies on software algorithms that detect and correct camera shake in the client device capturing the images. Examples of such algorithms are for instance described in the article “Using Optical Flow for Stabilizing Image Sequences” from the author P. O'Donovan, and the article “Fixation as a Mechanism for Stabilization of Short Image Sequences” from the authors K. Pauwels, M. Lappe, and M. M. Van Hulle. Since these algorithms perform video stream processing at the client side, these solutions are not device-agnostic and therefore cannot guarantee a ubiquitous, coherent video conference experience across all devices. Moreover, these algorithms typically occupy substantial processing power and memory resources in the client devices of the conference participants.
It is an objective of the present invention to provide a method and tool for immersive video conferencing that resolves the above-mentioned disadvantages of existing solutions. More particularly, it is an objective of the present invention to disclose an immersive video conferencing solution that allows to improve the overall quality of immersive video conferencing by detecting, signalling and correcting camera shake in a manner that is agnostic to the client devices used by the conference participants, and that can be implemented with limited requirements in terms of usage of processing power and memory.